Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update snapshot-restore.md #4655

Merged
merged 3 commits into from
Oct 17, 2023
Merged

Update snapshot-restore.md #4655

merged 3 commits into from
Oct 17, 2023

Conversation

sandervandegeijn
Copy link
Contributor

Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Description

Describe what this change achieves.

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@sandervandegeijn
Update snapshot-restore.md
6ad5a6e 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
@kolchfa-aws
Copy link
Collaborator

Thanks, @ict-one-nl! @bowenlan-amzn Could you tech review this PR please?

@kolchfa-aws kolchfa-aws self-assigned this Jul 31, 2023
@kolchfa-aws
Copy link
Collaborator

@jed326 Would you be able to review this PR?

@jed326
Copy link
Contributor

jed326 commented Aug 10, 2023

@kolchfa-aws sorry for the late response.

I still see this setting in the code: https://github.com/opensearch-project/OpenSearch/blob/e3542011cd9584c354405ba6cbf9a31ced7bc5ce/plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java#L238-L243

@ict-one-nl can you provide some more details of how you did the setup?

@sandervandegeijn
Copy link
Contributor Author

Okay, it's kind of weird;

Config:

#s3.client.default.disable_chunked_encoding: false # Disables chunked encoding for compatibility with some storage services, but you probably don't need to change this value.
s3.client.default.endpoint: scality02.xxxx.nl:443 # S3 has alternate endpoints, but you probably don't need to change this value.
s3.client.default.max_retries: 3 # number of retries if a request fails
s3.client.default.path_style_access: true # whether to use the deprecated path-style bucket URLs.
#You probably don't need to change this value, but for more information, see https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html#path-style-access.
s3.client.default.protocol: https # http or https
s3.client.default.read_timeout: 50s # the S3 connection timeout
s3.client.default.use_throttle_retries: true 
s3.client.default.region: us-east-2 #lijkt verplicht

Remove the comment on s3.client.default.disable_chunked_encoding:

image

image

Comment it out:

image

everything is fine

@msfroh
Copy link
Contributor

msfroh commented Oct 3, 2023

The reason why the exception occurs is because the plugin doesn't register support for the setting:

https://github.com/opensearch-project/OpenSearch/blob/main/plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java#L203-L229

Interestingly, if I go back to 1.0, it wasn't there either:

https://github.com/opensearch-project/OpenSearch/blob/1.0/plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java#L109-L129

Based on the commit where the setting was added: opensearch-project/OpenSearch@07cf2cb, it looks like you can't add it as a default for all repositories (i.e. as a cluster-level setting), but you can specify disable_chunked_encoding in the body of a repository-creation request.

@msfroh
Copy link
Contributor

msfroh commented Oct 3, 2023

@ict-one-nl , are you able to confirm whether the setting works as part of a repository creation request?

Regardless, I agree with this specific PR -- it shouldn't be documented as an opensearch.yml setting.

@msfroh
Copy link
Contributor

msfroh commented Oct 3, 2023

I think we should probably document the disable_chunked_encoding setting on https://opensearch.org/docs/latest/api-reference/snapshots/create-repository/

@kolchfa-aws
Copy link
Collaborator

Documented the disable_chunked_encoding setting in #5242

@kolchfa-aws kolchfa-aws mentioned this pull request Oct 17, 2023
Merged
1 task
kolchfa-aws
kolchfa-aws approved these changes Oct 17, 2023
View reviewed changes
Copy link
Collaborator

@kolchfa-aws kolchfa-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @ict-one-nl!

@kolchfa-aws kolchfa-aws added backport 1.3 PR: Backport label for v1.3.x backport 2.0 PR: Backport label for v2.0.x backport 2.1 PR: Backport label for 2.1 backport 2.2 PR: Backport label for 2.2 backport 2.3 PR: Backport label for 2.3 backport 2.4 PR: Backport label for 2.4 labels Oct 17, 2023
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Oct 17, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
1d9d461 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This was referenced Oct 17, 2023
Merged
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
4c3c10b 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Oct 17, 2023
Merged
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5243)
e715506 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
4bdbc26 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Oct 17, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
4a6ad1d 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
22a4764 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This was referenced Oct 17, 2023
Merged
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
b952fd2 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Oct 17, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 17, 2023
@github-actions @kolchfa-aws
Update snapshot-restore.md (#4655)
a05bb37 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
(cherry picked from commit 17977e4)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5244)
53a79c7 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5245)
6e4d879 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5246)
169ab50 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5247)
dcfc32f 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5248)
6e13cde 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5249)
5b03514 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5250)
98134df 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5251)
58ab994 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5252)
174866d 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5253)
06d2926 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5254)
18e9014 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
kolchfa-aws added a commit that referenced this pull request Oct 17, 2023
@opensearch-trigger-bot @github-actions @kolchfa-aws
Update snapshot-restore.md (#4655) (#5255)
2fdaa15 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.



(cherry picked from commit 17977e4)

Signed-off-by: Sander van de Geijn <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: kolchfa-aws <[email protected]>
harshavamsi pushed a commit to harshavamsi/documentation-website that referenced this pull request Oct 31, 2023
@sandervandegeijn @kolchfa-aws @harshavamsi
Update snapshot-restore.md (opensearch-project#4655)
b482247 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
vagimeli pushed a commit that referenced this pull request Dec 21, 2023
@sandervandegeijn @kolchfa-aws @vagimeli
Update snapshot-restore.md (#4655)
3f52e09 
Setting seems deprecated. With it present clusters fails to start with message it does not recognise this setting.

Signed-off-by: Sander van de Geijn <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kolchfa-aws kolchfa-aws kolchfa-aws approved these changes

@msfroh msfroh msfroh approved these changes

@cwillum cwillum Awaiting requested review from cwillum

@hdhalter hdhalter Awaiting requested review from hdhalter

@Naarcha-AWS Naarcha-AWS Awaiting requested review from Naarcha-AWS Naarcha-AWS is a code owner

@vagimeli vagimeli Awaiting requested review from vagimeli

@ananzh ananzh Awaiting requested review from ananzh

@seanneumann seanneumann Awaiting requested review from seanneumann

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@natebower natebower Awaiting requested review from natebower natebower is a code owner

Assignees

@kolchfa-aws kolchfa-aws

Labels
backport 1.3 PR: Backport label for v1.3.x backport 2.0 PR: Backport label for v2.0.x backport 2.1 PR: Backport label for 2.1 backport 2.2 PR: Backport label for 2.2 backport 2.3 PR: Backport label for 2.3 backport 2.4 PR: Backport label for 2.4 backport 2.5 PR: Backport label for 2.5 backport 2.6 PR: Backport label for 2.6 backport 2.7 PR: Backport label for 2.7 backport 2.8 PR: Backport label for 2.8 backport 2.9 PR: Backport label for 2.9 backport 2.10 PR: Backport label for 2.10 backport 2.11 PR: Backport label for 2.11
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sandervandegeijn @kolchfa-aws @jed326 @msfroh